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ABSTRACT 

An  intriguing  aspect  of  visualization  as  a  technology  is  that  it  offers  the  potential  to  improve  the  functional 
capacity  of  the  human  operator.  In  this  report,  we  discuss  how  this  can  be  done  in  a  variety  of  ways: 
by  recoding  information,  by  using  multimodal  displays,  by  training,  or  by  using  intelligent  interfaces, 
agents,  and  ontologies.  In  doing  so,  we  argue  that  the  capacity  of  the  human-machine  system  should  be 
defined  as  distributed  working  memory  -  at  information  shared  between  the  human  and  machine  when 
performing  a  dynamic  task.  We  also  discuss  how  the  system  design  process  can  be  structured  to 
incorporate  human  capacity. 


INTRODUCTION 

We  have  known  for  a  long  time  that  the  main  bottleneck  in  human  information  processing  is  not  how 
much  of  the  environment  we  can  perceive  using  our  various  sensory  systems,  nor  in  how  much  knowledge 
we  can  accumulate.  Rather,  the  tightest  constraint  lies  in  how  much  information  we  can  process 
simultaneously.  This  constraint  has  two  key  sources:  an  attentional  bottleneck  (e.g.,  Pashler,  1998),  and  as 
a  constraint  in  working  memory  (Baddeley,  1995)  or  limited  mental  resources  (e.g.,  Wickens,  1984). 
In  this  paper,  we  attempt  to  define  capacity  by  briefly  reviewing  the  literature  on  attention,  limitations  on 
our  information  processing,  chunking,  working  memory,  multiple  resource  theory,  and  long-term  working 
memory.  In  so  doing,  we  examine  the  effects  of  combining  modalities  (vision,  audition,  haptics, 
kinesthetics).  Then  we  consider  technology  as  a  method  for  increasing  capacity,  both  in  terms  of  improved 
interface  design  and  by  using  intelligent  systems.  We  present  a  model  of  task  demands  on  working 
memory,  leading  to  a  model  of  distributed  working  memory .  Finally  we  discuss  how  the  system  design 
process  can  be  structured  to  incorporate  human  capacity. 

Attentional  Constraints 

The  notion  of  attention  as  a  spotlight  is  a  useful  metaphor.  It  highlights  some  of  problems  we  have  with 
attention;  sometimes  the  spotlight  is  in  the  wrong  place  (problem  of  selective  attention),  sometimes  the 
spotlight  is  set  too  broadly  so  that  irrelevant  information  intrudes  (problem  of  focused  attention);  and 
sometimes  it  is  difficult  to  broaden  the  spotlight  sufficiently  to  process  all  task-relevant  information 
(problem  of  divided  attention)  (Wickens  &  Hollands,  2000).  Because  it  seems  most  relevant  to  the  process 
of  visualization,  we  consider  here  the  situation  where  a  task  cannot  be  performed  without  particular  task¬ 
relevant  information  being  displayed  to  the  operator.  In  this  context,  there  appear  to  be  two  fundamental 
problems  related  to  capacity  and  visual  attention.  The  first  is  that  attention  needs  to  be  allocated  to  task¬ 
relevant  source  or  sources.  If  this  fails,  and  no  task  relevant  information  is  identified,  the  operator  will  fail. 
To  the  extent  that  the  operator  can  identify  task-relevant  information  in  the  scene,  performance  will 
improve.  If  spatial  attention  is  allocated  to  inappropriate  data  sources  (the  problem  of  selective  attention) 
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performance  will  degrade.  This  may  happen  because  irrelevant  data  are  highly  salient,  for  example. 
In  some  cases,  the  operator  may  be  able  to  perform  the  task  even  if  not  all  task  relevant  information  is 
identified  (by  deduction  from  obtained  information,  for  example).  However  it  will  serve  to  increase  the 
amount  of  processing  that  working  memory  must  do  later.  If  attention  is  guided  using  specific  display 
techniques  (e.g.,  Muller  &  Rabbitt,  1989),  then  detection  of  task-relevant  information  may  be  aided.  Even 
the  best  sensor  data,  data  fusion  algorithms,  and  display  technology  will  not  aid  the  operator  if  the  data 
being  collected,  fused,  and  displayed  is  not  relevant  to  the  operator’s  task.  Visualization  software  is  most 
effective  when  it  presents  task-relevant  information. 

If  other  data  are  also  portrayed  with  task-relevant  information,  the  operator  should  still  be  able  to  perform 
the  task.  However,  this  leads  to  the  second  problem  for  capacity  and  visual  attention.  If  there  are  too  many 
data  sources,  performance  tends  to  degrade  given  attentional  limitations.  The  operator  must  filter  the 
irrelevant  data  from  the  relevant  data  (producing  the  problem  of  focused  attention  described  above), 
or  may  find  it  too  difficult  to  broaden  the  attentional  spotlight  sufficiently  to  attend  to  all  task-relevant 
information  sources  (producing  the  problem  of  divided  attention).  What  is  necessary  is  to  detect  task¬ 
relevant  information  in  the  environment  and  dynamically  allocate  attention  to  those  sources.  The  less 
irrelevant  data  there  are,  the  better  the  operator’s  performance.  Visualization  software  will  be  maximally 
effective  when  it  presents  task-relevant  information  only. 

Sometimes  however,  we  are  not  aware  of  the  location  of  the  task-relevant  information  and  need  to  search 
for  it.  In  visual  search  (where  an  observer  searches  for  a  particular  target  symbol  among  a  set  of  distractor 
symbols)  there  is  evidence  that  some  targets  can  be  found  very  quickly  with  no  penalty  for  an  increase  in 
the  number  of  distractors  (efficient  search,  or  pop  out)  (Wolfe,  1997).  For  example,  it  is  easy  to  find  the 
sole  red  symbol  in  a  field  of  blue  symbols.  In  contrast,  visual  search  for  other  symbols  is  less  efficient, 
with  search  time  increasing  with  the  number  of  distractors  (e.g.,  finding  an  O  in  a  set  of  Qs).  Careful 
formatting  and  well-designed  symbology  can  lead  to  more  efficient  search  on  maps  and  tactical  displays. 
Some  information  can  be  processed  with  little  attentional  demand,  whereas  efficient  search  reflects  a 
preattentive  capacity  -  bypassing  the  attentional  bottleneck  altogether  (Treisman  &  Gelade,  1980). 
In  contrast,  controlled  search  taps  attentional  resources  for  the  duration  of  the  search.  Thus,  efficient 
search  has  the  potential  to  increase  our  attentional  capacity  in  terms  of  bandwidth  -  reducing  the  time  to 
find  what  we  seek. 

In  summary,  effective  visualization  is  better  realized  when  task  relevant  information  is  presented  without 
extraneous  data,  or  when  the  task-relevant  information  is  made  more  salient.  If  search  is  necessary,  it  is 
better  to  make  the  target  of  that  search  preattentive,  by  using  unique  stimulus  levels  or  dimensions. 

Some  Constraints  on  our  Ability  to  Process  Information 

In  his  classic  paper,  Miller  (1956)  summarized  the  large  literature  on  absolute  judgment  by  showing  how 
our  capacity  to  classify  alternatives  was  relatively  constant  regardless  of  stimulus  type  (shapes,  sizes, 
tastes,  odors,  etc).  The  magical  number  was  approximately  7  alternatives  (7  ±  2).  In  the  absolute  judgment 
task,  an  observer  is  presented  with  a  set  of  stimuli  and  asked  to  classify  them  by  name  or  number.  As  the 
number  of  alternatives  in  the  set  is  increased,  so  does  the  information  that  a  human  observer  can  transmit, 
but  errors  start  to  occur  between  2  and  3  bits  (4  or  8  stimulus  alternatives),  with  an  asymptote  of  2.6  bits 
(7  alternatives). 

Later  work  combining  multiple  dimensions  (e.g.,  Egeth  &  Pachella,  1969)  showed  that  this  capacity  could 
be  increased  by  having  people  classify  alternatives  varying  on  multiple  stimulus  dimensions  (e.g.,  length, 
width,  and  color).  With  this  technique,  capacity  can  be  increased:  for  example  observers  can  transmit 
5.8  bits  (or  approximately  100  positions)  for  the  spatial  position  of  a  dot  in  a  square  (two  dimensions  of 
length  and  width).  As  the  number  of  dimensions  is  increased  beyond  two,  the  amount  of  information 
transmitted  asymptotes  at  about  6.8  bits. 
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The  nature  of  the  different  dimensions  being  combined  is  also  important  for  capacity.  Work  on  multiple 
resource  theory  showed  that  it  is  possible  to  perform  multiple  tasks  simultaneously  (good  timesharing) 
if  they  drew  upon  different  resource  pools  (Wickens  &  Hollands,  2000).  We  can  divide  attention  between 
eye  and  ear  better  than  between  two  auditory  or  visual  channels,  for  example.  We  can  also  perform 
simultaneously  a  verbal  task  and  a  spatial  task  better  than  two  verbal  tasks  or  two  spatial  tasks. 

In  related  work,  Baddeley  (1986,  1995)  proposed  that  working  memory >  (WM)  can  be  subdivided  into  a 
number  of  component  systems  (articulatory  loop  for  verbal/phonological  processing;  visuospatial 
sketchpad  for  visuospatial  processing,  and  a  central  executive  component).  There  is  also  evidence  for  an 
additional  kinaesthetic  component  used  for  maintaining  information  about  body  position  (Woodin  &  Heil, 

1996) .  Evidence  in  support  of  this  separation  of  WM  activity  is  provided  by  experiments  where 
participants  perform  multiple  tasks  simultaneously.  As  with  the  multiple  resource  experiments, 
interference  is  observed  when  two  tasks  draw  on  the  same  component  system,  but  not  when  the  tasks  draw 
on  different  systems.  In  summary,  results  from  multiple  resource  and  WM  research  indicate  that  capacity 
can  potentially  be  increased  beyond  the  6.8  bits  limit  if  we  take  advantage  of  different  sensory  modalities, 
or  different  types  of  processing  codes. 

Limits  on  WM  capacity  are  also  affected  by  stress.  In  particular,  although  moderate  arousal  levels  can  be 
beneficial  to  performance,  high  stress  reduces  working  memory  capacity  and  tends  to  produce  attentional 
narrowing  or  cognitive  tunnelling  (Wickens  &  Hollands,  2000).  The  user  tends  to  focus  on  specific 
characteristics  of  the  situation  and  has  difficulty  switching  the  focus  of  attention  (perseveration ). 
If  irrelevant  data  are  presented  in  a  salient  manner,  this  can  heighten  to  focused  attention  problem  for  the 
operator.  Increase  in  stress  levels  has  been  shown  to  interfere  with  situation  awareness,  also  (Orasanu, 

1997) .  It  appears  especially  important,  therefore,  to  find  display  techniques  that  present  all  task-relevant 
information  to  reduce  the  need  for  computation,  and  present  task-relevant  information  to  the  exclusion  of 
irrelevant  data  (to  reduce  the  need  for  attentional  filtering). 

It  is  evident  that  the  background  or  expertise  of  the  user  influences  processing  capacity.  For  example,  it  is 
well  known  that  when  a  procedure  becomes  well  learned  it  becomes  automatized',  that  is,  it  requires  fewer 
processing  resources,  thereby  making  it  easier  to  perform  that  task  with  another  task.  Thus,  expertise  in  a 
task  domain  offers  a  method  for  improving  performance  by  reducing  WM  demands.  A  second  advantage 
of  expertise  is  that  the  contents  of  WM  change.  Consider,  for  example,  the  chess  master  who  can 
memorize  the  position  of  chess  pieces  after  brief  presentation  (Chase  &  Simon,  1973).  Experts  can  group 
or  chunk  relevant  information  together  based  on  meaning  (e.g.,  chess  pieces  into  chess  positions),  allowing 
the  operator  to  maintain  much  more  information  in  WM  (Miller,  1956).  Thus,  WM  capacity  is  better 
indexed  by  chunks  than  bits.  This  offers  an  alternative  solution  to  increasing  WM  capacity.  Indeed,  these 
two  processes  work  together  so  that  using  WM  contents  to  access  long-term  memory  may  become 
automatized  with  sufficient  practice  or  training,  offering  a  long-term  working  memory  (LTWM)  with 
immense  capacity  (Ericsson  &  Kintsch,  1995).  We  speculate  that  since  Generation  Y  are  pre-trained  on 
many  basic  computer  interface  techniques  the  current  generation  of  soldiers  can  take  advantage  of  a  wider 
variety  of  innovative  interfaces  to  accomplish  a  task.  If  a  particular  response  device  (e.g.,  joystick) 
is  highly  automatized  for  a  member  of  Generation  Y,  its  use  will  not  tap  WM  resources  and  will  not 
preclude  good  performance  on  concurrent  tasks. 


FACTORS  AFFECTING  WORKING  MEMORY  LOAD 

We  argue  that  a  variety  of  factors  can  serve  to  reduce  demand  on  working  memory  resources.  These  are 
illustrated  in  Figure  1 ,  which  plots  working  memory  load  as  a  function  of  a  variety  of  capacity-enhancing 
strategies.  Each  vector  represents  a  particular  strategy,  which  we  outline  below.  Because  effective 
visualization  designs  would  probably  employ  the  various  strategies  in  combination,  we  represent  the  effect 
of  the  strategies  on  two  tasks,  A  and  B,  using  a  hyperplane  surface.  The  effect  of  each  strategy  is  to 
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decrease  working  memory  load,  although  different  strategies  may  have  differential  effects  for  particular 
tasks.  The  figure  depicts  Task  A  as  generally  requiring  greater  working  memory  resources  than  Task  B, 
although  the  use  of  capacity-enhancing  strategies  could  allow  WM  load  for  Tasks  A  and  B  to  be  equal. 

Working  Memory  Load 


Icons  Mods.* 


*  Input  Channel  &  Mode  Agility 
Data  Compression  Techniques 
Intuitive  Presentation  Management 
Case  Based  Reasoning 

Figure  1:  Working  Memory  Load  as  a  Function  of  Several  Capacity  Enhancing  Strategies. 

Familiarity  and  case-based  reasoning.  As  discussed  earlier,  working-memory  load  is  influenced  by 
familiarity.  Familiarity  on  a  task  has  three  key  benefits:  automaticity,  chunking,  and  automatized  long¬ 
term  memory  access  (LTWM).  Automaticity  is  typically  produced  from  repetitive  experience  (training)  in 
the  work  environment,  which  can  be  tremendously  useful  for  repetitive  tasks  but  is  also  highly  specific  to 
particular  situations  (Schneider  &  Shifffin,  1977).  The  use  of  LTWM  allows  for  recognition-primed 
decision-making,  which  asserts  that  decision  options  are  quickly  selected  on  the  basis  of  familiarity;  that 
is,  similarity  of  the  current  situation  (that  portrayed  on  the  display)  to  previously  experienced  situations 
(Klein,  1989).  The  model  in  Figure  1  was  also  influenced  by  Commanders’  (Cols  Johansen  and  Alward, 
2002  this  workshop)  remarks  regarding  the  need  for  familiarity  in  all  display  aspects  for  efficiency  and 
stress  reduction.  In  the  figure,  we  use  the  term  case-based  reasoning  to  indicate  more  directly  the  change 
in  the  nature  of  mental  processing  that  occurs  with  expertise.  We  view  this  as  a  greater  change  along  the 
familiarity  magnitude  because  it  is  more  flexible  and  of  broader  utility. 

By  finding  familiar  metaphors  in  visualization,  decision  making  or  problem  solving  can  be  reduced  to 
“an  already  encountered  problem”,  and  thus  dealt  with  in  a  known  way.  A  simple  example  would  be  to 
depict  abstract  non-spatial  data  using  a  familiar  spatial  technique  (e.g.,  depicting  intrusion  detection  data 
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by  plotting  file  transfer  activity  as  a  function  of  time  and  server  location  using  a  3D  surface  and  allowing 
the  user  to  fly  over  the  surface  looking  for  the  region  with  the  highest  mountains). 

Response  modality  also  plays  a  role  here.  By  taking  away  buttons,  handles  and  sticks,  and  moving 
controls  into  digital  displays  where  the  same  buttons  have  different  mode-dependent  meanings,  it  puts 
strain  on  WM  resources  and  does  not  allow  automaticity  to  develop. 

Input  channel  and  mode  agility.  Particular  input  channels  may  be  better  suited  to  particular  tasks.  For 
example,  there  is  a  well  recognized  relationship  between  mode  of  input  (visual  vs.  auditory)  and  the  nature 
of  the  person’s  output  (spatial  vs.  verbal),  part  of  the  concept  of  stimulus-response  compatibility  (Wickens 
&  Flollands,  2000).  Moreover,  an  implication  of  the  work  on  multiple  resource  theory  (Wickens  1984) 
is  that  capacity  can  be  enhanced  (i.e.,  working  memory  load  can  be  reduced)  when  two  tasks  draw  on 
different  resource  types.  This  has  implications  for  how  information  is  displayed  to  the  user.  Results  from 
absolute  judgment  experiments  indicate  that  using  multiple  stimulus  dimensions  can  increase  working 
memory  capacity  -  whether  this  advantage  is  greater  when  multiple  modalities  are  used  has  not  to  our 
knowledge  been  well  investigated.  Particular  individuals  may  be  better  able  to  switch  between  different 
modalities  as  they  perform  tasks,  an  attribute  we  call  mode  agility.  (We  speculate  that  Generation  Y  may 
have  particular  advantages  in  this  regard,  due  to  gaming  experience).  The  use  of  different  display 
modalities  would  presumably  be  more  effective  for  these  individuals.  Whether  this  is  trainable  is  also  not 
well  investigated  to  our  knowledge. 

The  nature  of  the  response  method  is  also  important  and  must  be  considered  in  visualization.  For  example, 
by  allowing  users  to  point  to  a  location  of  interest  on  the  battlefield  using  gesture  recognition,  little 
demand  is  placed  on  WM  (consider  Colonel  Johansen’s  idea  behind  “the  table”,  this  workshop). 
In  contrast,  by  designing  displays  in  such  a  way  that  users  forced  to  define  locations  verbally, 
WM  resources  are  diverted  to  an  unnecessary  task. 

Data  compression  techniques.  Most  sensors  collect  vast  quantities  of  data  and  their  output  needs  to 
summarized  and  co-ordinated  thru  various  data  fusion  techniques  (Waltz  &  Llinas,  1990).  We  argue  that 
such  techniques  can  serve  to  reduce  working  memory  load  relative  to  processing  data  in  rawer  form  and 
thereby  offer  a  tremendous  benefit  to  the  user.  For  example,  it  has  been  demonstrated  that  sonar  data  are 
best  presented  to  the  operator  after  filtering  and  other  integration  techniques  have  been  conducted. 

Intuitive  presentation  management.  Here  we  argue  for  visualization  systems  to  ensure  that  task-relevant 
information  is  displayed.  Information  should  correspond  to  the  mental  processing  required  to  perform 
task-related  activities  with  respect  to  a  particular  work  domain.  Further,  we  argue  that  intuitive  displays 
decrease  the  amount  of  mental  processing  required  to  perform  the  task,  reducing  working  memory  load. 
One  might  argue  that  the  computer  can  perform  the  necessary  computation  using  RAM  and  cached  storage 
so  that  the  user  does  not  use  limited  working  memory  resources  to  that  end.  For  example,  by  re-orienting 
the  viewpoint  location  on  a  three-dimensional  (3D)  geographic  terrain  model  the  user  can  perceive  the 
correspondence  between  terrain  elements  on  a  PDA  display  and  the  forward  field  of  view  (FFOV). 
In  contrast,  the  use  of  a  fixed  viewpoint  can  lead  to  left-right  reversals  with  respect  to  terrain  elements  and 
could  require  mental  rotation  to  determine  which  display  object  corresponds  to  objects  in  the  FFOV. 
Porathe  (2002,  this  workshop)  describes  a  similar  design  solution  for  a  display  on  a  ferry  bridge. 

This  strategy  may  also  be  enhanced  through  the  use  of  intelligent  agents  or  ontologies.  For  example, 
Zeltzer  (2002,  this  workshop)  demonstrated  software  that  depicted  planned  routes  through  terrain  given 
current  conditions  and  intelligence  information.  The  route  was  plotted  on  a  3D  terrain  model  and  could  be 
viewed  from  multiple  orientations. 

If  task-relevant  information  is  not  present  at  point  of  gaze,  the  user  must  search  for  it,  requiring  allocation 
of  attention  over  available  display  space,  and  may  also  require  command  input  to  the  computer  to  change 
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the  nature  of  the  displayed  information  to  address  task  requirements.  Both  activities  should  increase 
working  memory  load,  although  certain  display  characteristics  can  reduce  attentional  demand,  and  highly 
overleamed  command  sequences  can  be  issued  automatically  (without  controlled  processing).  In  the 
former  case,  the  search  for  certain  kinds  of  targets  among  certain  kinds  of  distractors  can  lead  to  highly 
efficient  search  in  which  the  number  of  objects  on  the  display  has  no  effect  on  search  time  (Wolfe,  1997). 
In  the  latter  case,  command  sequences  must  be  used  consistently  and  repeatedly  to  produce  automatic 
processing  with  training.  The  use  of  keyboard  commands  has  been  shown  to  be  more  efficient  than  long 
menu  searches  for  this  purpose  (Raskin,  2000). 

DISTRIBUTED  WORKING  MEMORY 

Consider  the  Turing  test  (Turing,  1950).  If  the  mechanism  for  intelligent  activity  is  hidden  from  the 
observer,  and  the  observer  believes  it  to  be  acting  intelligently,  it  does  not  matter  whether  the  behavior 
was  generated  by  a  human,  a  machine,  or  some  combination  thereof.  In  a  human-computer  or  human- 
machine  system,  we  argue  that  the  activity  of  working  memory  can  be  shared  between  the  information 
displayed  by  the  interface  (visual,  auditory,  haptic,  etc)  and  the  human  operator.  We  call  this  model 
distributed  working  memory  (DWM).  DWM  is  thus  conceived  of  as  task  relevant  displayed  information 
and  the  contents  of  a  user’s  working  memory. 

The  simplest  version  of  the  DWM  model  is  depicted  in  Figure  2.  Here,  DWM  is  depicted  as  a  stacked  bar 
whose  sum  represents  the  information  in  the  system.  At  the  top  of  the  bar  is  data  stored  in  the  computer  or 
in  the  world  via  sensor  information.  Below  that  is  information  portrayed  on  a  display  device,  including 
visual  displays,  auditory  displays,  or  kinaesthetic  information  from  controls  such  as  joysticks,  buttons, 
mice,  etc.  The  line  just  below  this  represents  the  interface.  Below  that  WM  contents  are  represented. 
At  the  bottom  is  a  large  long-term  memory  store  (LTM). 
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Figure  2;  A  Model  of  Distributed  Working  Memory  (DWM). 
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We  focus  on  two  capacity  limitations  in  this  system,  both  on  the  human  side.  The  first  capacity  limitation 
in  this  system  is  the  attentional  bottleneck.  Attention  is  portrayed  as  the  interface  of  working  memory  and 
information  on  the  display(s).  If  the  contents  of  the  display  are  highly  related  to  the  user’s  current  task, 
as  shown  in  Figure  2,  there  is  little  need  for  attentional  selection  or  broadening.  If  only  some  display 
contents  are  related  to  the  task,  as  shown  in  Figure  3,  there  is  need  for  attentional  selection.  If  task  relevant 
information  is  not  at  the  point  of  gaze,  visual  search  is  necessary,  which  may  require  attentional  resources. 
If  some  information  is  not  currently  available  in  the  display,  working  memory  resources  may  be  required 
to  issue  a  series  of  commands  to  the  computer  to  change  the  displayed  information  to  correspond  (unless 
the  command  sequence  is  automatized). 
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Figure  3:  DWM  as  a  Function  of  Changes  in  the  Allocation  of  Attention. 


The  second  capacity  limitation  is  WM  capacity.  Performing  a  task  will  typically  require  WM  resources. 
Units  of  WM  capacity  are  chunks,  to  allow  for  domain  expertise  due  to  recoding  or  training.  For  example, 
a  commander  may  be  monitoring  the  altitudes  of  and  distances  between  multiple  aircraft  over  time  using  a 
tactical  display.  If  the  commander  knows  that  the  formation  being  flown  specifies  relatively  fixed 
distances  between  aircraft,  one  value  can  be  maintained  in  working  memory  rather  than  multiple  values, 
freeing  up  WM  resources.  If  the  orientation  of  two  tactical  displays  is  inconsistent,  forcing  the  observer  to 
mentally  rotate  one  or  the  other  (or  to  make  L-R  reversals)  this  will  require  greater  WM  resources  than  if 
the  two  displays  are  aligned.  If  information  from  a  surveillance  aircraft  indicates  that  all  altitudes  were 
incorrect  and  need  to  be  doubled,  extra  computation  will  be  required  every  time  the  observer  must 
communicate  altitude  information  to  other  members  of  the  command  team,  demanding  WM  resources. 

We  assume  computer  memory  capacity  is  infinite,  or  at  least  so  large  with  respect  to  human  WM  that 
increases  in  displayed  information  have  minimal  effect  on  the  processing  capacity  of  the  computer. 
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Intuitive  presentation  management  can  be  accomplished  using  dynamic  memory  allocation  -  that  is,  better 
designed  or  intelligent  interfaces  can  perform  some  of  the  computation  that  otherwise  the  human  would 
do,  freeing  up  WM  resources.  As  noted  above,  intelligent  interfaces  and  ontologies  that  depict  planned 
routes  through  terrain  given  current  conditions  and  intelligence  information  serve  this  role.  Effective  data 
fusion  and  compression  techniques  can  similarly  be  effective.  Consider  the  system  developed  by  Jungert  et 
al.  (2002,  this  workshop).  Their  design  ensures  that  it  is  not  necessary  for  the  user  to  specify  the  precise 
level  of  sensor  data  to  formulate  a  query. 

Figure  4  depicts  an  augmentation  to  the  model  to  allow  for  different  processing  resources  as  a  function  of 
modality.  We  have  discussed  the  notion  of  a  WM  subsystem,  identified  a  distinction  between  spatial  and 
verbal  processing,  and  noted  that  a  verbal-spatial  task  combination  will  be  performed  better  than  a  verbal- 
verbal  or  spatial-spatial.  Similarly,  we  may  consider  that  our  mental  resources  are  differentiable  by 
modality  (e.g.,  visual,  auditory,  tactile)  or  processing  stage  (Wickens,  1984).  Another  way  of  thinking 
about  the  shift  from  physical  buttons  handles  and  sticks  to  virtual  controls  described  earlier  is  that  we  have 
shifted  from  taking  advantage  of  a  different  (output,  or  late  processing  stage)  to  drawing  on  already 
overtaxed  WM  resources. 
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Figure  4;  DWM  Modified  to  Incorporate  Multiple  Sensory  Modalities. 


DESIGN  RECOMMENDATIONS 

Figure  5  asserts  that  the  capacity-enhancing  aspects  of  any  visualization  design  is  a  “system”  problem. 
The  box  at  the  top  of  the  figure  describes  components  of  system  design  that  must  be  considered  with 
respect  to  the  human  user,  the  work  domain,  and  the  task.  The  design  problem  starts  with  the  assessment 
of  the  human  role  in  any  computer-aided  system,  and  that  is  at  the  “Functional  Allocation”  step  in  which 
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the  human-machine  roles  are  defined  and  characterized.  The  second  stage  is  to  prototype  the  system 
incoiporating  information  gained  from  cognitive  task  analysis  and  cognitive  work  analysis  approaches, 
which  allow  better  understanding  of  the  user’s  task  and  work  domain,  respectively.  The  achievement  of 
intuitive  presentation  management  (central  oval  in  the  figure)  depends  on  consideration  of  the  many 
design-influencing  factors  surrounding  that  oval.  If  these  components  are  incorporated  into  the  design 
process,  the  result  should  be  a  system  that  allows  the  user  to  visualize  task-relevant  information  in  a  quick, 
efficient  way. 

Human  Capacity  &  Total  System  Design 


1.  Functional  Allocation 

-  Initial  Human  Capacity  Specification 

2.  System  Design  &  Prototyping 

-  Visualization  Subsystem 
3  Test  and  Evaluation 

-  Capacity  Assessments 


*  e.g.  see  Zeltzefs  brfg. 
•*  e  g  see  Jungert’s  brfg. 


Figure  5;  Capacity-Enhancing  Strategies  Incorporated  into  Design. 


RESEARCH  RECOMMENDATIONS 

We  make  the  following  recommendations  for  research  topics: 

1)  Examine/explore  whether  capacity  can  be  increased  by  using  coding  dimensions  from  different 
modalities 

2)  Determine  how  can  we  design  systems  that  adapt  to: 

a)  Limitations  of  the  human— Increasing  “capacity”  by  design:  navigation  engines  etc. 

b)  Flow  of  information  to  correspond  to  situational  demands 

3)  Examine  whether  Generation  Y  has  increased  mental  capacity  given  their  early  exploration  of 
computers. 
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SYMPOSIA  DISCUSSION  -  SYNDICATE  3 


Comment: 

With  expertise  and  learning,  people  can  develop  long-term  memory  that  can  function  as  working  memory. 

Comment: 

It  would  be  interesting  to  integrate  the  findings  of  vast  numbers  of  small  experiments  that  have  been  done 
to  compile  the  evidence  that  exists  in  areas  such  as  human  cognitive  architecture,  as  well  as  parallel  and 
motor  human  possessors. 

Question: 

Research  that  suggests  that  different  people  think  in  different  ways,  should  the  same  modalities  be 
presented  to  every  user,  or  can  you  exploit  the  capabilities  of  the  individual? 

Response: 

Research  into  the  system  learning  about  the  capabilities  of  the  user  and  knowledge  the  user’s  preferences 
and  abilities  and  provide  info  accordingly. 
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Problem  Statement 


Problem:  how  can  we  use  visualization  to 
overcome  human  capacity  limitations 


Findings 

•  Background:  Defining  capacity 

-  Updating  7±2  chunks 

-  Effect  of  combination  of  modalities  (vision,  audition,  haptic, 
kinesthetic) 

•  Generation  “I”:  pre-trained 

•  Better  displays;  displays  that  don’t  require  mental 
transformation 

•  “Don’t  give  me  symbology” 

•  Dataflood  problem 

•  Recognition-primed  decision  making 

•  Intelligence:  navigation  engines,  agents,  ontology 

•  Teams  and  collaboration:  Trade-off  between  comm  and  vis 


S3-3 


Working  Memory  Load 


Icons  Mods.* 


*  Input  Channel  &  Mode  Agility 
Data  Compression  Techniques 
Intuitive  Presentation  Management 
Case  Based  Reasoning 
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Human  Capacity  &  Total  System  Design 


1.  Functional  Allocation 

-  Initial  Human  Capacity  Specification 

2.  System  Design  &  Prototyping 

-  Visualization  Subsystem 

3.  Test  and  Evaluation 

-  Capacity  Assessments 


*  e.g.  see  Zeltzer's  brfg. 
**  e.g.  see  Jungert's  brfg. 
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Research  Recommendations 


Examine/explore  whether  capacity  can  be 
increased  by  using  coding  dimensions  from 
different  modalities 

How  can  we  design  systems  that  adapt  to: 

-  Limitations  of  the  human— Increasing  “capacity”  by 
design:  navigation  engines  etc. 

-  Flow  of  information  to  correspond  to  situational 
demands 

Examine  whether  Generation  “I”  has  increased 
mental  capacity  given  their  early  exploration  of 
computers 


